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P^ , Abstract. We compute the limit distribution for (centered and scaled) length of 

*vj ' the longest increasing subsequence of random colored permutations. The limit distri- 

bution function is a power of that for usual random permutations computed recently 



Ctf 



Ctf 



by Baik, Deift, and Johansson ( math. CO/9810105 ). In two-colored case our method 



\m/ , provides a different proof of a similar result by Tracy and Widom about longest 

^J ' increasing subsequences of signed permutations ( math. CO/981 1154 ). 



pH Our main idea is to reduce the 'colored' problem to the case of usual random 

permutations using certain combinatorial results and elementary probabilistic argu- 
ments. 



(N 

> ; 1. Introduction 

^h ! 

Baik, Deift, and Johansson recently solved a problem about the asymptotic 
behavior of the length l n of the longest increasing subsequence for the random 

permutations of order n as n — > oo (with the uniform distribution on the symmetric 

ON 

On 



group S n ). They proved, see [BDJ], that the sequence 

l n - 2y/n' 



n 



1/6 



converges in distribution, as n — > oo, to a certain random variable whose distribu- 
/\! tion function we shall denote by F(x). This distribution function can be expressed 

via a solution of the Painleve II equation, see [BDJ] for details. It was first obtained 
by Tracy and Widom [TW1] in the framework of Random Matrix Theory where it 
gives the limit distribution for the (centered and scaled) largest eigenvalue in the 
Gaussian Unitary Ensemble of Hermitian matrices. 

The problem of the asymptotics of l n was first raised by Ulam [U] . Substantial 
contributions to the solution of the problem have been made by Hammersley [H] , 
Logan and Shepp [LS], Vershik and Kerov [VK1, VK2]. 

A survey of the interesting history of this problem, further references, and a 
discussion of its intriguing connection with Random Matrix Theory can be found 
in [BDJ]. 

Soon after the appearance of [BDJ] Tracy and Widom computed the asymp- 
totic behavior of the length l' n of the longest increasing for the random 'signed 
permutations', see definitions in the next section. In [TW2] they showed that 

C-2%/2^ 

2 2/3( 2n )l/6 



converges in distribution, as n — » oo, to a random variable with the distribution 
function F 2 (x). 

The present paper provides another proof of the result by Tracy and Widom. In 
our approach the distribution function F 2 (x) arises as the distribution function of 
the maximum of two asymptotically independent variables each of which behaves 
as (l n — 2y / n)/n 1 ' 6 (hence, by [BDJ], converges to the distribution given by F(x)). 

The combinatorial techniques we use relies on recent works by Rains [R] and 
Fomin & Stanton [FS]. It also allows to handle a more general case of 'colored 
permutations' (the problem for 'two-colored case', essentially, coincides with that 
for signed permutations). We show that for the length Z" of the longest increasing 
subsequence of the random m-colored permutations of order n the sequence 



l' n -2^n~n 

m 2/3( TOn )l/6 

converges in distribution, as n — ► oo, to a random variable with distribution function 
F m (x). The function F m (x) naturally appears as the distribution function of the 
maximum of m asymptotically independent variables, each having F(x) as the limit 
distribution function. 

Combinatorial quantities which we consider can be also interpreted as expecta- 
tions of certain central functions on unitary groups, see Section 4. 

I am very grateful to G. I. Olshanski for a number of valuable discussions. 

2. Colored permutations and signed permutations 

A colored permutation is a map from {1, . . . , n} to {1, . . . , n} x {1, . . . , m} such 
that its composition with the projection on the first component of the target set is 
a permutation (of order n) . One can view such a map as a permutation with one 
of m colors attributed to each of n points which this permutation permutes. The 
set of all colored permutations of order n with m colors will be denoted by Sn ■ 

An increasing subsequence of n G Sn is a sequence 1 < i\ < %2 < ■ ■ ■ < ik < n 
such that the first coordinates of n(ij) increase in j and the second coordinates of 
Tv(ij) are equal. Thus, elements of an increasing subsequence are of the same color, 
say, p. The length of such increasing subsequence is defined to be m(k — 1) + p. 

These definitions are due to Rains [R]. A slightly more general notion of hook 
permutation was introduced and intensively used earlier by Stanton and White 
[SW]. 

We shall consider Sn as a probability space with uniform distribution: proba- 
bility of each colored permutation is \Sn | _1 = (m n n!) -1 . Then the length of the 
longest increasing subsequence becomes a random variable on this space, it will be 
denoted as L n ° . 

Let H n be the hyperoctahedral group of order n defined as the wreath product 
Z2 x S n (S n is the symmetric group of order n). The elements of H n are called 
signed permutations. This group can be naturally embedded in S2 n as the group 
of permutations a of {— n, —n + 1,...,— 1,1, ...,n — l,n} subject to the condition 
a(—x) = —a(x). Indeed, each such permutation is parametrized by the permutation 
\a\ G S n and the set of signs of cr(l), . . . , o~(n). Using the natural ordering on the 
set {— n, —n + 1, . . . , — 1, 1, . . . , n — 1, n} we can define the length of the longest 
increasing subsequence for each signed permutation. Assuming that every signed 
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permutation has probability |i/ n | _1 = (2 n ro!) _1 , we get a random variable on H n 
which will be denoted as L^ 6n . 

The group H n can also be embedded into the symmetric group of order 2n + 1: 
we add to the set {— n, — n + 1, . . . , —1, 1, . . . , n — 1, n} and assume that the 
elements a G H n satisfy the same condition cr(— x) = —a(x). Clearly, this implies 
cr(0) = 0. The random variable on H n equal to the length of the longest increasing 
subsequence with respect to this realization will be denoted by L° dd . 

Note that for any element a G H n , 

L° n dd (a)-L^(a)=0orl. (2.1) 

3. Rim hook tableaux 

We refer to the work [SW] for the definitions concerning rim hook tableaux. 
The next claim is a direct consequence of the Schensted algorithm, see [S] . 

Proposition 3.1. Permutations of order n with the length of longest increasing 
subsequence equal to I are in one-to-one correspondence with pairs of standard 
Young tableaux of the same shape with n boxes and width I. 

Here is a generalization of this claim for colored permutations. 

Proposition 3.2 [R], [SW]. Colored permutations with m colors of order n with 
the length of longest increasing subsequence equal to I are in one-to-one correspon- 
dence with pairs ofm-rim hook tableaux of the same shape with ran boxes and width 
I. 

In [SW] it was proved that [— ] = [— ] where w is the width of the rim hook 
tableau corresponding to a permutation with the length of longest increasing sub- 
sequence equal to / ( [a] stands for the smallest integer > a) . The refinement of 
this statement given above was published in [R]. 

Proposition 3.3 [R]. Signed permutations of order n embedded in S2n with the 
length of longest increasing subsequence equal to I are in one-to-one correspondence 
with pairs of 2 -rim hook tableaux of the same shape with 2n boxes and width I. 

The length of the longest increasing subsequence for signed permutations em- 
bedded into the symmetric group of odd order can also be interpreted in terms of 
rim hook tableaux, see [R, proof of Theorem 2.3]. 

Note that Propositions 3.2 and 3.3 imply that the distributions of random vari- 
ables L°n and L e ^ en coincide. 

4. Expectations over unitary groups 

Everywhere below the symbol "E,u eu {k) f {U) stands for the integral of / over 
U G U(k) with respect to the Haar measure on the unitary group U(k) normalized 
so that E{/ e [/(fc)l = 1 (i.e., E denotes the expectation of / with respect to the 
uniform distribution on the unitary group). 

Proposition 4.1 [R]. 

Prob{L^ m ) <k} = (m n nl)- 1 ■ E UeU(k) (\Tr(U m ) n \ 2 ) . (4.1) 



Proposition 4.2 [R]. 

Prob{Lr n <k} = (2 n nl)- 1 ■ E UeU(k) (\Tr{U 2 ) n \ 2 ) . (4.2) 

Prob{L° n dd <k} = (2-n!)" 1 • E UeU(k) (\Tr{U 2 ) n Tr{U)\ 2 ) . (4.3) 

[DS] gives (4.1) for k > mn, (4.2) for k > 2n, and (4.3) for k > 2n + 1. For such 
values of k the left-hand sides of (4.1), (4,2), (4.3) are all equal to 1. 

5. Rim hook lattices 

Our main reference for this section is the work [FS] by Fomin and Stanton. 

For this section we fix an integer number m, all our rim hooks here will contain 
exactly m boxes. 

Let fi and A be shapes (Young diagrams) such that /iCA and A — \x is a (m-)rim 
hook. Then we shall write \i /* A. 

We introduce a partial order on the set of Young diagrams as follows: A y \x 
if and only if there exists a sequence v\, v^-, ■ ■ ■ , Vk of Young diagrams such that 
^/^/^/••■/^/^ The empty Young diagram is denoted by 0. We 
shall say that a Young diagram A is m-decomposable if A y 0. 

The poset of all m-decomposable shapes with y as the order is called rim hook 
lattice and is denoted by RH m . (It can be shown that this poset is indeed a lattice). 

For m = 1 we get the Young lattice: the poset of all Young diagrams ordered by 
inclusion. The Young lattice will be denoted by Y. 

Proposition 5.1 [FS]. The rim hook lattice RH m is isomorphic to the Cartesian 
product of m copies of the Young lattice: RH m = Y r 
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In other words, RH m is isomorphic to the poset of m-tuples of Young diagrams 
with the following coordinate-wise ordering: one tuple is greater than or equal to 
another tuple if the kth coordinate of the first tuple includes (i.e., greater than or 
equal to) the kth coordinate of the second tuple for all k = 1, . . . , m. 

Clearly, the number of m-rim hook tableaux of a given shape A is equal to the 
number of paths /* v\ /* v^ /* • • • /* Vh /* A, k = \X\/m — 1, from to A (and 
is equal to if A is not m-decomposable), |A| stands for the number of boxes in A. 
We shall denote this number by dim m A and call it the m-dimension of the shape 
A. 

Take any A G RH m and the corresponding m-tuple (Ai, . . . , A m ) G Y m . Note 
that |A| = m(|Ai| + • • • + |A m |). We have 

dim m A = — \—r r- — pp— -dimiAi-'-dimi A m . (5.1) 

|Ai|! • • • |A m |! 

Indeed, to specify the path from to (Ai, . . . , A m ) we need to specify m paths 
from to Afc in the kth copy of Y for k = 1, . . . , m together with the order in 
which we make steps along those paths. The number of different orders is the 
combinatorial coefficient in the right-hand side of (5.1) while the number of different 
possibilities for the m paths in Y is the product of 1-dimensions dimi Ai • • • dimi A m . 

Note that the number of pairs of m-rim hook tableaux of the same shape A is 
exactly 

A- 2 X f (|Al| + '--+|A m |)! \, 2 , 2> ,. 2 X ,. v 

dim m A = p— r: j- — j-: • dm^ Ai • • • dm^ A m . (5.2) 



Ai !••• A 



i 
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Let us denote by w(X) the width of a Young diagram A. We shall need the 
following 

Observation 5.2. For any A G RH m and corresponding m -tuple (Ai, . . . , A m ) G 
Y m we have 

m ■ max{w(Ai), . . . , w(X m )} — w(X) G {0, 1, . . . , m — 1}. (5-3) 

This follows immediately from the explicit construction of the isomorphism from 
Proposition 5.1, see [FS, §2]. 

Observation 5.2 will be crucial for our further considerations. 

6. Plancherel distributions 

Using the correspondence from Proposition 3.2 (which is exactly the rim hook 
generalization of the Schensted algorithm, see [SW]), we can associate to each 
m-colored permutation of order n a Young diagram with ran boxes - - the com- 
mon shape of the corresponding pair of m-rim hook tableaux. The image of the 
uniform distribution on Sn under this map gives a probability distribution on m- 
decomposable Young diagrams with mn boxes; the weight of a Young diagram A is, 
clearly, equal to (m n n\)~ 1 ■ dim m A. As a consequence, we get (cf. [FS, Corollary 
1.6]) 

J2 dim 2 m X = m n nl. (6.1) 

|A|=mn 

Using the isomorphism of Proposition 5.1 we transfer our probability distribution 
to the set of m-tuples of Young disgrams with total number of boxes equal to n. 
Then by (5.2) we see that the probability of an m-tuple (Ai, . . . , A m ) G Y m with 

| Ai| + • • • + |A m | = n equals 

1 / ! \ 2 



Prob{(Ai,...,A m )} = -^- 7 ' ) •dimfA 1 ---dimfA m . (6.2) 

lib IV. \ AJ . ' ' ' A^ — ' 



v m • 



This distribution will be called the Plancherel distribution. 
Proposition 6.1. For any n = 1, 2, . . . 

Prob{|Ai| = ni,...|A m | = n m ;ni H h n m = n} 

- JL nl (6 - 3) 

m n nil ■ ■ -n m ! 

Proof. Direct computation. Using (6.1) for m = 1 and (6.2) we get 
Prob{|Ai| = ni,...|A m | = n m ;ni H \- n m = n} 

y^ ( TTTi — ~R — iT ) ' dim i ^1 " ' dim i ^™ 
\|Ai|!- • • |A m |!/ 



m n n\ 

|Afc|=n fc 
fc=l,...,m, 

1 / n! A 2 1 n! 

■ml-'-riml = — — -. □ 



m n n\ \n\\ ■ • • n m \ J ' m n n\\---n m \ 



7. TWO LEMMAS FROM PROBABILITY THEORY 

We shall denote by — >■ convergence of random variables in probability, and by 
— ► convergence in distribution. 

Lemma 7.1 [B, Theorem 4.1]. Let random variables £, {£,n}iZi, {Vn}iZi satisfy 

£n ~ Vn -^ 0, £ n > £. 

Then 

Vn — ► £. 

For to real random variables £i , • • • , £m we shall denote by £i x £2 x • • • x £ m a 
M m -valued random variable with distribution function 

f {ix-x{ ra W = F^ixx) ■ ■■F u (x m ). 

Lemma 7.2. Let {£n }J£Li, fe = l,...,m ; 6e m > 1 sequences of real random 
variables convergent in distribution to random variables £( 1 ), . . . ,^ m \ respectively. 
Denote by B n m> the nth order m- dimensional fair Bernoulli distribution: 

(m) _ (u u \\_ J ^ fci!"'fc m !' ki + --- + &m = ™, &i <E {0,...,7^} 



Prob{i?r ; =(fcl,•••,U} , _ 

U, otherwise 



j i 



Then the sequence 



C - t(i) x t(2) x ... x cH 

Sn — S„(m) A S B (m) A A S B (m) 

n,l n,2 n,m 



converges in distribution to t;^ x £( 2 ) x • • • x £( m ). /n particular, £ {m) — ► £W 



pi.'") 

71,1 



Proof. The conditions £n — ► £^ , fc = 1, . . . , m, are equivalent to the pointwise 
convergence of distribution functions 

F (fe) (x) -^Fe(k)(x). 

Sri ^ 

We have 

fclH hfc m =n 

fci = l,...,n 

For to convergent number sequences {a n } with limits a' fc ), fc = 1, . . . , m, the 
sequence 



1 n! 
a^' •••« 

TO 

fciH hfc m =n 

fcj=l,...,n 



V a (1) a (m) 



converges to the product a^ 1 ) • • -aS™', so we conclude, that {F^ n (x)} converges to 
Ft (xi) • • • F^ (x m ) pointwise. D 

Lemma 7.2 implies that random variables £ (m) , £ (rre) , . . . , £ (j^ are asymptot- 
ically independent as n — > 00. 



8. ASYMPTOTICS 

Baik, Deift, and Johansson recently proved, see [BDJ], that the sequence 

L c n ol(1) - 2^ 



n 



1/6 



converges in distribution to a certain random variable, whose distribution function 
will be denoted by F(x). This function can be expressed through a particular 
solution of the Painleve II equation, see [BDJ] for details. 

In this section we shall study the asymptotic behavior of Ln° for m > 2 when 
n — ► oo. Our main result is the following statement. 

Theorem 8.1. For any m = 2, 3, . . . the sequence 

L c n ol{m) - 2^hH 



(mn) 1 / 6 

converges in distribution, asn — ► oo, to a random variable with distribution function 
F m (m~3x). 

This result for m = 2 was proved by Tracy and Widom in [TW2]. Since the 
distributions of L°n and L^ en coincide (see Section 3), we immediately get two 
other asymptotic formulas. 

Corollary 8.2 [TW2]. The sequence 



L even _ 2 ^ 2n 

(2^)V6 

converges in distribution, as n — *■ oo , to a random variable with distribution function 
F 2 {2~lx). 

Corollary 8.3 [TW2]. The sequence 

L°^ d - 2 v / 2^ 
(2n) 1 /6 

converges in distribution, asn — > oo, to a random variable with distribution function 
F 2 {2~lx). 

Proof of Corollary 8.3. Relation (2.1) implies that with probability 1, 

(L° n dd - L e n ven ) (2n)" 1/6 -»■ 0. 

Since the convergence with probability 1 implies the convergence in probability, the 
claim follows from Lemma 7.1 and Corollary 8.2. □ 

Proof of Theorem 8.1. The proof will consist of 4 steps. 

Step 1. Following Sections 3, 5, 6, we shall interpret Ln° as a random variable 

on the space of all m-tuples (Ai,...,A m ) of Young diagram with total number 
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of boxes equal to n supplied with the Plancherel distribution. Let us denote by 
/ j n , . . . , lm the widths of the Young diagrams Ai, . . . , A m : 

/^ n) =w(X k ), k= l,...,m. 
Observation 5.2 implies that with probability 1 

L C n (m) - m ■ max{Zp , . . . , lf%' } 

n^ " 

Hence, by Lemma 7.1, it is enough to prove that 

. m ■ max-fjp, . . . , V&>} - 2^[ma , , _ ^ , 

Prob < — — tttt— l < x > ->■ F m (m s x ) 

1 (mn) 1 /6 - ' 

Our strategy is to prove that l^ 1 , . . . , lm are asymptotically independent, and 
that each of them asymptotically behaves as L^°,i. Then Theorem 8.1 will follow 
from the result of [BDJ] stated in the beginning of this section. 
Step 2. 

_, . .m-iad^{l{ l \...,lm}-2^/ma^ 

Prob < - —7^ < x 

{ran) 1 ' 



. m ■ l\ — 2 J ran m ■ l m — 2 J ran 

Prob ^ 1 uTe x ! • • • 5 ? uTe x 

{ran) 1 ' {ran) 1 ' 

/[ n) - 2^/n/ra 2 Z^ - 2 v / n/m 2 

Prob<; — ? / u/6 — - m3a; ;---; — 7 / u/6 — < m31 ^- 

Thus, it suffices to prove that 
„ , f l\ n) -2Jn~hn~ 1$ -2Jn~hn~ } , s , s , s 

Prob 1 —( i Ti/6 <gi;---; r / u/6 <» m ->%-%■ 8.1 

Step 3. Denote by n\ n the number of boxes in Aj, i — 1, . . . , m. Then n^ + • — h 
n-m = n. We claim that for alH = 1, . . . , m 



(»)_o./^T / W -^/ n (n) 



/r-2yW^ n-zyn, „ 



(n/m)V6 ( n (-)) 1/6 



Proposition 6.1 implies that 



Prob{n^=lfc} = . (ro - 1)n "^ n 



(n) 

This means that n\ can be interpreted as the sum £i + • • • + £ n of n independent 
identically distributed Bernoulli variables with 

ra — 1 I 

Probte = 0} = , Probfe = 1} = -• 

ra m 



The central limit theorem implies that with probability converging to 1, 



i+e 



n ( n \ 2+ £ ( n \ n 

--(-) < n i <- + [- 
to \mJ m \m 



n \ h+ e 



for any e > 0. Hence, 







(-)* 


- («!"') 


i 

3 


< 


\mJ 


i 

( U _u ( n \^ +£ \ a 

^m Vto/ J 




= (-)' 

Vto/ 


-(-©" 4+ f 


( n \ ~h+ e 
< const ■ — I 
Vto/ 


for sufficiently large n. 




Similarly, we have 






u -^ } 6 


< 


vto/ 


((-)±(-Y +e Y h 

\\m/ Vto/ J 




f - 


H 


m) 


;N 


i 

r 


< const ■ 


/ n \ ~2~6+ £ 

— < const • 
Vto/ 



.3) 



if we choose e < 1/12. 

The result of [BDJ] implies that L 

From Proposition 6.2 it follows that in the notation of Lemma 7.2 



col(l) —1—S "D n C x ^ n 

'n 2 — ► tor any o > 0. 



n \ ~\~ e 



(8.4) 



in) 



n-1 



T col(l) 

\ n.,1 ) J n=1 



Applying Lemma 7.2 we see that 



M) 



(n)\? +S 



V 



n\ 

which means that l\ n < [n™ ) with probability converging to 1. 

Since n\ n < n/m + (n/m)^~ with probability converging to 1, we get that 



,(»)<( n + /nx^'" 



< 



to vto 
n\h+ s 
m 



n\2' 
m< 



TO/ 



n\ s +^ 



. - fn\h+ 5 

+ const ■ ( — I < const ■ - 



with probability converging to 1. 

Making use of the relations (8.3), (8.4), (8.5), we now see that 



(8.5) 



t ] -2^nM t } -2jn^ 



+2 



(n/m) 1 / 6 



n\a ( n ) i 

— ) -K 3 

TO/ 



K ) 



</ 



(n) 



n\ e M,-i 

— -K 6 

m/ 



< const 



n \ 2 



TO 



i+5 



+ const 



n \ ~6 



m 



+ £ 



with probability converging to 1. Since the last expression converges to zero as 
n — > oo if 5 < £ < 1/6, the proof of (8.2) is complete. 

The relation (8.2) implies that we have asymptotic equivalence of two m-di- 
mensional vectors 



4 n) -2v^M_ 4 n) -2v^7 



III 



(n/m) 1 / 6 '"■■' (n/m)V6 



(»)_o</,>) iWoJ.W 



/r-2V<' ; C j -2\/n 



5.6) 



?j 



(4"») 1/6 '"" (r4">) 1/6 
Step 4. The random m-dimensional vector 



0. 



(n^) V6 ; '" ; (nfc>) 
is obtained from m identical 1-dimensional random variables 



Cn ~ ' (n),V6 -•- (n) ,l/6 I ( 8 - ? ) 



7-00/(1) _ n /- 



„l/6 ' -'•••' 

by the procedure described in Lemma 7.2: 

c _t(i) x t(2) x ... x tH 

"n,\ D n,2 D n,"i 

(this follows from Proposition 6.1). Since sequences {£n } converge in distribution, 
as n — > oo, to a random variable with distribution function F(x) (this is the result of 
[BDJ]), the sequence (8.7) also converges in distribution to a random variable with 
the distribution function F(xi) ■ ■ •F(x m ). Then Lemma 7.1 and (8.6) conclude the 
proof of (8.1). □ 
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